State of the art video codecs, such as the joint video specification from the International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) ISO/IEC 14496-10 AVC and the International Telecommunication Union-Telecommunication Standardization Sector (ITU-T) H.264 (commonly referred to as H.264/AVC), Society of Motion Picture and Television Engineers (SMPTE) VC-1, Moving Pictures Expert Group (MPEG) MPEG-4 Visual and ITU-T H.263, employ block-based transform coding. An image (i.e., a still image, original video pictures or motion compensated video pictures) is first divided into blocks. Each block is then transformed from a spatial domain to a frequency domain using an 8×8 Discrete Cosine Transform (DCT), a 4×4 DCT, an 8×8 integer transform or a 4×4 integer transform. Each of the resulting DCT/integer transform coefficients is quantized with a given quantization stepsize. The quantization stepsizes, or a scaled version of the stepsizes corresponding to individual DCT/integer transform coefficients, may be represented using a matrix. Under the JPEG standard, the matrix is called a “quantization matrix”. The matrix is referred to as a “quantization weighting matrix” in MPEG-1/2/4, H.261 and H.263. The matrix is referred to as a “quantization scaling matrix” in MPEG-4 and H.264/AVC. Hereafter, the matrix is called a quantization matrix (QM).
Referring to FIG. 1, a block diagram of a conventional block-based image/video system 60 is shown. The system 60 includes an encoder 62 and a decoder 64. The boxes T, Q and C in the encoder 62 may represent a forward DCT, a forward quantizer and an entropy encoder respectively. The boxes T−1, Q−1 and C−1 in the decoder 64 may represent an inverse DCT, an inverse quantizer and an entropy decoder respectively. Data L represents a quantization matrix, which is used for both encoding and decoding. Where the encoder 62 implements a JPEG encoding, the quantized DCT coefficients (i.e., J(x,y)) with a horizontal and vertical spatial frequencies of x and y is given by equation 1 as follows:J(x,y)=R(H(x,y)/L(x,y))  Eq. (1)where R is an integer rounding function and H represents transform coefficients. At the decoder 64, the quantized coefficients are inverse quantized by equation 2 as follows:H′(x,y)=J(x,y)×L(x,y)  Eq. (2)where H′ represents reconstructed transform coefficients.
The use of a quantization matrix in encoding and decoding allows different weighting to be applied according to a sensibility of the human visual system to a corresponding frequency of a coefficient. Since the human visual system is less sensitive to high spatial frequencies than to low spatial frequencies, the higher orders of DCT coefficients are quantized with coarser quantized step sizes than the lower frequency coefficients. The higher the horizontal or the vertical index, the larger the quantization matrix element.
A conventional principle of guiding a design of a quantization matrix is to provide optimum visual quality for a given bitrate. A common method of designing quantization matrices is to use a visibility of quantization errors at the various DCT frequencies. For each spatial frequency pair (x,y), a smallest coefficient that yields a visible signal is measured. The smallest coefficients sets a threshold (i.e., T(x,y)) for all DCT coefficients at (x,y). Since a maximum possible quantization error at (x,y) is L(x,y)/2, the quantization matrix is set to ensure that all errors are invisible per equation 3 as follows:L(x,y)=2×T(x,y)  Eq. (3)
For a given image, the DCT quantization errors may be adjusted by contrast sensitivity, light adaptation or contrast masking. The DCT quantization errors are also pooled non-linearly over the blocks of the image. Based on the resulting perceptual error matrix, an image dependent quantization matrix is designed.
Another conventional approach for generating quantization matrices is to model the matrices. Since an 8×8 quantization matrix has 64 coefficients and a 4×4 quantization matrix has 16 coefficients, a search performed in a 64-dimensional space or a 16-dimensional space of optimal coefficient sets is computationally prohibitive. Modeling the quantization matrices using a few parameters is a feasible way to reduce dimensions of a solution space. A conventional three parameter model of a quantization matrix is given by equation 4, shown in FIG. 2, where A, B and C are the three parameters. The quantization matrix in equation 1 is modeled as symmetric matrix where elements along the arcs √{square root over (x2+y2)} are constant. The model reduces significantly the dimension of a quantization matrix from 16 coefficients or 64 coefficients to 3 parameters. However, evaluations of the square root function and the sine function are expensive. Furthermore, equation 4 can over-simplify the quantization matrix space with only three parameters.